Hand Held Machine Vision Method and Apparatus

ABSTRACT

A method and system for use with a portable device including a processor, at least some embodiments of the method including the steps of providing an identification tag on at least one item that includes item identifying information, obtaining item identifying information from the identification tag via the portable device, obtaining at least one image of the at least one item via the portable device and performing a supplemental process using the identifying information and the at least one image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent is a continuation-in-part of U.S. patent application Ser.No. 11/020,640 which was filed on Dec. 22, 2004 and which has the sametitle.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

BACKGROUND OF THE INVENTION

The present invention relates to machine vision and more specifically toportable machine vision systems for collecting images of items or groupsof items, identifying the item(s) and performing one or more secondaryfunctions using the collected images.

When assembly lines were first configured to increase the rate ofmanufacturing processes, often the results of processes had to bemeasured by hand to ensure high quality products. For instance, wheretwo parts had to be between two and 2⅛th inch apart, a line worker hadto use some type of mechanical hand held measuring device to manuallymeasure the distance between the two parts and confirm that the distancewas within the required range. Similarly, where a bottle was to befilled to a specific height with detergent prior to shipping, a lineworker would have to manually examine the bottle to ensure that thedetergent level was within a required range. While manual inspection andmeasurement can have good results when performed correctly, in manycases such procedures were prone to error, were mundane for the personsperforming the procedures, were relatively expensive and time consumingto implement and could only be performed on a random basis (i.e., onevery 100th product that passed along the line).

More recently, camera or machine vision systems have been developed thateliminate many of the problems associated with manual inspectionprocedures. For instance, in one application a camera and a light sourcemay be rigidly mounted adjacent a station along a manufacturing transferline including clamps that, when a semi-finished product is moved to thestation, clamp the product in a precise juxtaposition (i.e., at aprecise distance from and along a precise trajectory to) with respect tothe camera and light source. Here, the camera and light source may bepositioned twelve inches from and normal to a space between first andsecond product components such that an image of the clamped productgenerated by the camera shows the first and second components separatedby the space having a space dimension. In this case, the image may beprovided to a processor for identifying the space dimension and forcomparing the identified dimension to a required space dimension betweenthe first and second components.

To determine the space dimension in the image, the processor isprogrammed to scale the space defined by the first and second componentsin the image appropriately assuming the precise juxtaposition of thecamera to the components (i.e., assuming a normal camera trajectory andtwelve inches between the camera and the space). Where the identifieddimension is different than an expected and required dimension theprocessor may be programmed to reject the part, to store the difference,to indicate the difference, to suggest a correction to eliminate thedifference, etc.

Other imaging applications in addition to dimension determinationinclude color verification, defect detection, object/patternverification, object recognition, assembly verification and archivalprocesses for storing images and/or other correlated information (e.g.,the results of color verification processes, defect detection processes,etc.). In each of the applications listed above, camera images areprocessed in a different fashion to facilitate different functions butthe precise juxtaposition restrictions required to generatemeaningful/useful images still exist.

Systems like the one described above work well in the context ofmechanical components that ensure specific relative juxtapositions of acamera, a light source and a product or product feature being imaged.Hereinafter systems that rigidly mount cameras and light sources inspecific positions with respect to other components that restrictproduct position during imaging will be referred to as “constrainedvision systems”.

Unfortunately, there are a huge number of manufacturing and otherapplications where constrained vision systems are not easily employedbut where the advantages of machine vision described above (i.e.,accuracy, speed, low cost, consistency, etc.) would be useful. Forinstance, in the case of jet engine manufacturing, engines are typicallyassembled in relatively small numbers at assembly stations. Here, anengine often includes a large number of different components and thedimensions and spacings between many of the components must be precisein order for the engine to operate properly. Measurement of each of therequired dimensions and spacings would require an excessive number ofstationary cameras and light sources and would therefore be too costlyfor most applications. In addition, even if costs associated withrequired stationary cameras and sources were not an issue, placement ofthe cameras and sources adjacent the assembly station and at properpositions for obtaining required images would be impossible given thefact that assembly personnel need to move freely within the assemblystation space during the assembly process.

Moreover, in many applications the product (e.g., a jet engine) beingassembled may be mounted for rotation about one or more axis so thatassembly personnel can manipulate the product easily to obtain differentoptimal vantage points for installing components. Here, where productposition is alterable, even if a stationary camera were providedadjacent an assembly station, precise positioning of the camera withrespect to the product would be difficult to achieve as the product ismoved to optimize position for installation purposes. Where cameraposition with respect to the product/feature being imaged is unknown oris inaccurately assumed, resulting images can be mis-scaled such thatcomparison to required ranges is inappropriate. For instance, assumethat a required distance between two features must be within a range offive and 5⅛th inches and a machine vision system assumes that a camerais twelve inches from and normal to a space between the two featureswhen an image is obtained. In this case, when an image is obtained, thesystem may be programmed to identify the two features and measures thedimension there between. Thereafter, the system scales the measureddimension assuming the image was taken from twelve inches and at anangle normal to the space between the two features. In this case, if acamera used to obtain an image is at an angle (e.g., 15 degrees) withrespect to normal and/or is fifteen inches from the space as opposed totwelve inches, the dimension calculated by the system will be differentthan the actual dimension and an error will likely be indicated even ifthe actual dimension is within the required range.

Another exemplary application where constrained vision systems are noteasily employed but where the advantages of machine vision describedabove would be useful is in small manufacturing plants. For instance, ina small metal working facility there may be 1000 different measurementsthat routinely need to be taken on various products. Here, while atleast some of the 1000 measurements may need to be made hundreds oftimes each year, the volume of product associated with even the mostroutine measurements may not justify the costs associated with even asingle stationary machine vision system. In these cases the advantagesof machine vision are typically foregone and all measurements end upbeing manual.

Thus, it would be advantageous to have a system and methods whereby theadvantages of machine vision systems could be employed in manyapplications in which such advantages have heretofore been foregone forvarious reasons and where costs associated therewith can be reducedappreciably. More specifically, it would be advantageous to have asystem and methods whereby one camera and light assembly could be usedto obtain many different product/feature images of one or a plurality ofproducts and where the system could automatically identify producttypes, image types and specific features associated with image andproduct types and could then perform product and image specificapplications or supplemental processes. Furthermore, it would beadvantageous to have a system and methods that provide guidance to asystem operator for generally optimally positioning a camera/lightsource with respect to products and features for obtaining suitableimages for processing.

BRIEF SUMMARY OF THE INVENTION

It has been recognized that many of the advantages associated withmachine vision in the context of constrained vision systems above can beextended to other applications by providing a camera and light source ona hand held portable device (hereinafter an “HHD”). Here, the HHD can bepositioned in any relative juxtaposition with respect to a product orproduct feature to obtain images thereof for analysis by systemsoftware. Where many different product characteristics need to be imagedfor examination purposes, the camera can be positioned in severaldifferent positions to obtain the images. In at least some cases systemsoftware can be programmed to recognize an obtained image as similar toa specific one of a plurality of optimal stored images that includesspecific features of interest so that, when the image is obtained, thesystem can automatically perform different processes on the imageinformation.

In at least some cases the HHD may include a guidance mechanism forhelping a user position the HHD with respect to a product or feature forobtaining an image suitable for specific processes. In some casesguidance may be provided by simply transmitting a specific light patterntoward a product to show the field of camera view. In other cases theguidance may be “active” and include indications that direct an HHD userto move the HHD in specific directions relative to the product orfeature being imaged. Where active guidance is provided, the HHD mayhave access to optimal product images stored in a database. Here, theHHD may obtain an image, compare the image to one or more of the optimalimages to identify positional changes that would result in a moreoptimal positioning of the HHD and then provide guidance to move left,move right, move up, move down, move forward, move backward, adjustpitch, adjust roll, etc.

Where the HHD is to be used to obtain images of two or more differentproduct types, an identification tag (e.g., bar code, RFID tag, etc.)may be placed on each of the products with information useable toidentify the product and the HHD may also be equipped with a tag readerfor obtaining information form the tags. In this case, prior to or afterobtaining an image of a product, the HHD may be used to obtaininformation from the tag so that the product type can be determined. TheHHD may be programmed to perform different functions for each differentproduct type. For instance, for a first product type the HHD may beprogrammed to guide the HHD user to obtain two different images that aresimilar to optimal stored images and to perform various measurements offeatures in the images while, for a second product type the HHD may beprogrammed to guide the HHD user to obtain five different images fromvarious orientations and to perform various measurements of features inthe five images. After the tag information is obtained the HHD canperform product type specific processes and provide product typespecific instructions.

Consistent with the above comments, at least some embodiments of thepresent invention include a method for use with a portable deviceincluding a processor, the method comprising the steps of providing anidentification tag on at least one item that includes item identifyinginformation, obtaining item identifying information from theidentification tag via the portable device, obtaining at least one imageof the at least one item via the portable device and performing asupplemental process using the identifying information and the at leastone image.

In addition, some embodiments of the invention include a method for usewith a database and a portable device including a processor, the methodcomprising the steps of (a) storing information in the databaseregarding at least one optimal relative juxtaposition of the portabledevice with respect to a first item type to be imaged, (b) positioningthe portable device proximate an instance of the first item, (c)obtaining at least one intermediate image of the first item instance; an(d) providing guidance information indicating relative movement of theportable device with respect to the first item instance to move theportable device toward the at least one optimal relative juxtapositionwith respect to the first item instance.

Here, in some cases the method may further include the step of, prior tostep (d), examining the intermediate image to identify position of theportable device with respect to the first item instance, the step ofproviding guidance including providing guidance as a function of theintermediate image examination. Steps (b) through (d) may be repeateduntil the portable device is at least substantially in the at least oneoptimal relative juxtaposition with respect to the first item instance.

At least some embodiments of the invention include a method for use witha database and a portable device including a processor, the method forobtaining information associated with a subset of a plurality ofdifferent item types, the method comprising the steps of providing anidentification tag on at least one item that includes item identifyinginformation, obtaining item identifying information from theidentification tag via the portable device, identifying the item typefrom the item identifying information, obtaining at least one image ofthe at least one item via the portable device, identifying at least onesupplemental process to be performed on the obtained image wherein thesupplemental subprocess is identified at least in part as a function ofthe item type and performing the at least one supplemental process onthe obtained image.

In addition, according to some inventive aspects, some embodimentsinclude a method for use with a database and a portable device includinga processor, the method for performing functions associated withdifferent items, the method comprising the steps of providingidentification tags on each of a plurality of items, each tag includingidentifying information useable to specifically identify an associateditem, obtaining item identifying information from at least oneidentification tag corresponding to a specific item via the portabledevice, obtaining other information associated with the specific itemincluding at least one image of the specific item via the portabledevice, performing supplemental processes on the other informationassociated with the specific item, monitoring the portable device for atransition indication and, when a transition indication is received,halting the supplemental processes.

Some embodiments include a system for use with items that includeidentification tags where each tag includes item identifyinginformation, the apparatus comprising a portable housing. a processormounted within the housing, a tag information obtainer supported by thehousing for obtaining information from the tags and a camera supportedby the housing, wherein, the processor runs a program to, wheninformation is obtained from a tag via the obtainer and an image isobtained via the camera, perform a supplemental process on the image asa function of the information obtained from the tag.

Moreover, some embodiments of the invention include an apparatus for usewith items that include identification tags where each tag includes itemidentifying information, the apparatus comprising a portable housing, aprocessor mounted within the housing, a tag information obtainersupported by the housing for obtaining information from the tags and acamera supported by the housing, wherein, the processor runs a programto, when information is obtained from a tag associated with a specificitem via the obtainer, perform supplemental processes associated withthe specific item on images obtained by the camera until a transitionindication is received and, when a transition indication is received,halting the supplemental processes associated with the specific item.

Furthermore, some embodiments include a method for use with a portabledevice including a processor, the method comprising the steps ofidentifying at least one item using the portable device, obtaining atleast one image of the at least one item via the portable device andperforming a supplemental process on the at least one image as afunction of the identity of the item.

According to one aspect the invention includes a method for marking aproduct for use with a camera including a field of view, the methodcomprising the steps of providing an identification tag on the productwherein the tag is machine readable to obtain information associatedwith the product and providing a source mark on the product spatiallyproximate the tag such that both the tag and the source mark can besimultaneously located within the camera field of view.

In addition, some embodiments include a method for use with a portabledevice including a processor, the method comprising the steps ofassociating an identification tag with at least one item that includesitem identifying information, obtaining item identifying informationfrom the identification tag via the portable device, obtaining at leastone image of the at least one item via the portable device andperforming a supplemental process using the identifying information andthe at least one image. Here, the step of associating may includeproviding the identification tag on the item or providing the tag in abooklet along with human distinguishable information associated with theitem.

These and other objects, advantages and aspects of the invention willbecome apparent from the following description. In the description,reference is made to the accompanying drawings which form a part hereof,and in which there is shown a preferred embodiment of the invention.Such embodiment does not necessarily represent the full scope of theinvention and reference is made therefore, to the claims herein forinterpreting the scope of the invention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a schematic view of an exemplary system including a handheldportable identification tag reader and camera device according to atleast some aspects of the present invention;

FIG. 2 is a schematic diagram illustrating some of the components thatform the handheld device of FIG. 1;

FIG. 3 is a perspective view of the handheld device of FIG. 1 being usedto obtain information from an identification tag according to at leastsome aspects of the present invention;

FIG. 4 is a flow chart illustrating an exemplary method according to thepresent invention;

FIG. 5 is a perspective view of a handheld device like the oneillustrating in FIG. 1 being used to obtain an image of an item forperforming any of several different supplemental processes according tovarious aspects of the present invention;

FIG. 6 is a flow chart illustrating a subprocess that may be substitutedfor a portion of the method of FIG. 4;

FIG. 7 is a an exemplary database according to at least some aspects ofthe present invention;

FIG. 8 is a flow chart illustrating another exemplary method accordingto at least some embodiments of the present invention;

FIG. 9 is an exemplary database consistent with at least some aspects ofthe present invention;

FIG. 10 is a plan view of an exemplary feedback arrangement that mayform part of the handheld device of FIG. 1;

FIG. 11 is similar to FIG. 8, albeit illustrating another methodaccording to at least some embodiments of the present invention;

FIG. 12 is another exemplary feedback arrangement according to at leastsome aspects of the present invention;

FIG. 13 is a screenshot for providing HHD guidance information accordingto at least some aspects of the present invention;

FIG. 14 is similar to FIG. 13, albeit providing other guidanceinformation;

FIG. 15 is similar to FIG. 14, albeit providing other guidanceinformation as well providing a real time image and a phantom showing anoptimal image orientation;

FIG. 16 is similar to FIG. 15, albeit providing other guidanceinstructions;

FIG. 17 is similar to FIG. 15, albeit providing results of asupplemental process after an optimal image has been obtained;

FIG. 18 is similar to FIG. 15, albeit providing alternate instructionsfor obtaining either of two different optimal images;

FIG. 19 is similar to FIG. 18, albeit providing results after an optimalimage has been obtained;

FIG. 20 is similar to FIG. 19, albeit providing results after adifferent optimal image has been obtained;

FIG. 21 is a partial plan view showing a source mark and anidentification tag on an item;

FIG. 22 is a flow chart illustrating a method for verifying partsaccording to at least some aspects of the present invention;

FIG. 23 is a flow chart illustrating a method for associating a portabledevice with a product for processing; and

FIG. 24 is a schematic diagram illustrating an HHD being used to obtainan image of a kit including various components according to at leastsome aspects of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

One or more specific embodiments of the present invention will bedescribed below. It should be appreciated that in the development of anysuch actual implementation, as in any engineering or design project,numerous implementation-specific decisions must be made to achieve thedevelopers' specific goals, such as compliance with system-related andbusiness related constraints, which may vary from one implementation toanother. Moreover, it should be appreciated that such a developmenteffort might be complex and time consuming, but would nevertheless be aroutine undertaking of design, fabrication, and manufacture for those ofordinary skill having the benefit of this disclosure.

Referring now to the drawings wherein like reference numerals correspondto similar elements throughout the several views and, more specifically,referring to FIG. 1, the present invention will be described in thecontext of an exemplary information system 10 including a work station12, a server 13, a database 14, a wireless access point 16 and ahandheld portable devices (HHD) 20. Work station 12 is linked to server13 for two-way communication via a network generally identified bynumeral 18. In addition, server 13 is linked via network 18 to accesspoint 16 as well as to database 14 for two-way communication therewith.Server 13 runs programs 15 stored in database 14 to perform variousmethods according to the present invention which will be describedhereinafter.

In addition to storing programs 15 run by server 13, database 14 alsostores information in a product/function database 17 corresponding toitems or products for which some process or method may be performed bysystem 10. In addition, in at least some embodiments of the presentinvention, database 14 also stores results or at least a subset of theresults in a results database 19 that occur after programs 15 have beenperformed.

In at least some embodiments of the present invention, interface 12includes a display screen 22 and some type input device such as keyboard24. Other types of input devices are contemplated including a mouse, atrack ball, voice recognition hardware, etc. Using work station 12, asystem operator can perform various tasks, depending upon the methodbeing performed by server 13, to facilitate method steps to be describedhereinafter.

Access point 16, as well known in the art, is a wireless transceiverthat is capable of receiving wireless signals (e.g., 802.11b, bluetooth, etc.) and transmitting similarly coded wireless informationwithin the area surrounding access point 16. When access point 16receives information, the information is decoded and transmitted toserver 13 via network 18. Similarly, server 13 can transmit informationwirelessly within the space surrounding access point 16 by transmittingthe information first via network 18 to access point 16 and causingaccess point 16 to transmit the information within the surrounding area.

Referring still to FIG. 1 and also to FIG. 2, HHD 20 includes a housing26 having a pistol type shape and includes a grip shaped handle 27 foreasy gripping via a user's hand and a barrel 29 that extends away from auser when handle 27 is gripped. An activation button in the form of atrigger 30 is provided on handle 27 where a users index finger residesduring normal use. Within housing 26 a plurality of components aremounted including, among other things, a processor 21, a camera 23, anHHD memory device 25, a light source 11 and a camera lens 28. In atleast some embodiments HHD 20 also includes a separate tag reader 400,some type of feedback device 32, 39, and/or a transceiver 34, shownlinked to processor 21 via phantom lines to indicate that at least someembodiments do not include these or a subset of these components.Processor 21 is linked for two-way communication to memory device 25 forstoring programs and information therein and retrieving program code andstored information when required. In addition, where HHD includes atransceiver 34, processor 21 is linked for two-way communication totransceiver 34 so that processor 21 can communicate with server 13 viaaccess point 16 and network 18 to receive information therefrom andprovide information thereto.

Referring still to FIG. 2, processor 21 is linked to activationbutton/trigger 30 for receiving activation signals therefrom. Processor21 is linked to light source 11 for controlling activation thereof. Inat least some cases source 11 will include two or more white light LEDsthat shine light through lens 28 (see again FIG. 1) when trigger 30 isactivated to illuminate a space in front of lens 28. In some cases whentrigger 30 is activated the LEDs generate a short flash of light. Insome cases it is contemplated that the light source 11 may be capable ofgenerating continuous light over a longer duration (e.g., 20 seconds).Processor 21 is also linked to camera/sensor 23 for receiving imagesgenerated thereby.

Referring still to FIG. 1, camera lens 28 is at the front end of barrel29 and has a field of view identified by numeral 42. In the embodimentillustrated in FIG. 1, transceiver 34 is mounted near the top end ofbarrel 29 to facilitate relatively unobstructed communication withaccess point 16. Feedback configuration 32, 39 (39 is a speaker) ismounted on the top surface of barrel 29 to facilitate feedback to an HHDoperator.

Referring to FIG. 3, an exemplary product, a jet engine rotor blade 200,is illustrated where the blade includes various features (e.g., edges)and feature characteristics (e.g., dimensions, curvatures, colors,etc.). For instance, blade 200 includes edges F1 through F12, top,middle and bottom thickness T1, T2 and T3 between edges F1 and F2, edgesF3 and F4 and edges F5 and F6, respectively, a length dimension Lbetween edges F7 and F8, a width dimension W between edges F9 and F10and a curvature between edge portions F11 and F12. Hereinafter, whileHHD 20 may be, and indeed is intended to be, used with many differentproduct/item types in at least some applications, HHD operations will bedescribed in the context of exemplary blade 200. In FIG. 3, anidentification tag 202 is provided on blade 200 that includes aconventional bar code. Other identification tag types are contemplatedincluding 2D bar codes, dot matrices, RFID tags, optical characterrecognition type tags, etc.

Referring now to FIG. 4, an exemplary simplified and general method 110according to at least some aspects of the present invention is shown. Inthe case of this first example, it is assumed that HHD 20 does notitself include a feedback configuration 32 and that, in fact, HHD 20 maybe used independent of the other system components illustrated inFIG. 1. Referring also to FIGS. 1 and 2, at block 108, a database 17 isprovided that correlates item identification information andsupplemental item related functions.

At block 111, HHD 20 is provided that includes a camera/tag reader asdescribed above. At block 112, identification tags are provided on itemsincluding blade 200 in FIG. 3 to be used with system 10. Continuing, atblock 114, a system user uses HHD 20 to obtain identificationinformation from one of the ID tags 202. At block 116, HHD 20 is used toobtain an image of the item associated with the most recently obtainedtag information. For instance, referring once again to FIG. 3 and alsoto FIG. 5, where HHD 20 is used to obtain ID information from tag 202 atblock 114 where tag 202 resides on blade 200, at block 116, HHD 20 isused to obtain an image of blade 200 (see FIG. 5). At block 117, HHDprocessor 21 identifies the item associated with tag 202 in database 17and the correlated supplemental function(s). At block 118 processor 21performs the supplemental function(s) using the obtained image. Thus,for instance, where the supplemental function is simply to store theobtained image with item identifying information, processor 21 may storethe obtained image correlated with the identifying information in HHDmemory 25.

According to other embodiments of the present invention, instead ofrequiring a system user to separately obtain an item image andidentification information from a tag 202 or the like, a single imagemay be obtained which includes features of an item that are of interestas well as the tag information. For instance, referring once again toFIG. 5, in at least some cases HHD 20 may be useable to obtain a fullimage of blade 200 including information specified by tag 202. Here,after an item image has been obtained, processor 21 may be programmed toisolate the region of interest of the obtained image including the tag202, read the tag information and then perform a supplemental process onthe image as a whole that is associated with the identification taginformation.

Consistent with the comments in the preceding paragraph, referring nowto FIG. 6, an exemplary subprocess that may be substituted for a portionof the process of FIG. 4 is illustrated. Referring also to FIG. 4, afterID tags are provided on items to be used with system 10, HHD 20 may beused to obtain a image of an item where the image includes the ID tag onthe item as well as other item features. Next, at block 130, HHDprocessor 21 identifies the ID tag (e.g., 202 in FIG. 6) in the imageand reads the identification information thereon. After block 130,control passes back to block 117 in FIG. 4 where the process describedabove continues.

In addition to storing correlated images and item/product identifyinginformation, other supplemental functions are contemplated that may beperformed via HHD 20. For instance, in at least some cases, it iscontemplated that HHD 20 may perform programs to analyze obtained imagesto identify regions of interest in the images, visually distinguishableproduct features within regions of interest and characteristics of thosefeatures. For example, referring again to FIG. 3, HHD 20 may beprogrammed to, when tag information indicates that an item associatedtherewith is a rotor blade 200, analyze the obtained image to identifytop and bottom edges F7 and F8, respectively, measure the lengthdimension L between edges F7 and F8, and store the length dimension Lcorrelated with the blade identifying information in HHD memory 25.Similarly, any of the other blade dimensions T1, T2, T3, W or C may becalculated and stored.

As another example, where optimal product dimensions have been specifiedin a database (e.g., in memory 25), HHD processor 21 may be programmedto, after item/product dimensions have been calculated, compare thecalculated values to the optimal values and store results of thecomparison in a correlated fashion with item identifying information inHHD memory 25.

Referring still to FIG. 1, where HHD 20 includes a transceiver 34, someor at least a portion of processing described above and hereinafter maybe performed by server 13. For instance, tag identification informationand images may be wirelessly transmitted from HHD 20 to server 13 viaaccess point 16 and network 18 so that server 13 can perform theitem/product identifying process, identify supplemental functions andperform supplemental functions.

Similarly, in at least some cases, HHD 20 may simply obtain tagidentifying information and product images, correlate and store theidentifying information and images and download the correlatedinformation to server 13 in batch either wirelessly or via a hard wireconnection (e.g., a USB port), additional supplemental processes beingperformed thereafter via server 13.

Where work station 12 or at least display 22 is proximate the locationof blade 200 during the imaging process and server 13 performs at leastparts of the supplemental processes, server 13 may provide at least asubset of the process results to an HHD user via display 22. Forinstance, product images may be provided via display 22 as well asproduct identifying information (e.g., “rotor blade type 00-0001”). Asanother instance, where product features and characteristics arecalculated from an image, the calculated values may be provided viadisplay 22 either in a list form or, where an obtained image isgenerated, as markings on the generated image. As still one otherinstance, where calculated values are different than expected or optimalvalues, the differences may be indicated via display 22 in some fashion.Moreover, where calculated values are within an expected or acceptablerange, an affirmation of the values as acceptable may be provided viadisplay 22. The above feedback functions may also beperformed/facilitated via display 22 where HHD processor 21 performsmost of the supplemental functions and then transmits results to thedisplay via point 16 and network 18.

Referring now to FIG. 7, an exemplary, albeit simplified,product/function database 17 is illustrated that includes three separatecolumns, an item ID No. column 78, an image features column 86 and afeature function column 92. Item ID No. column 78 lists a separateidentification number for each one of the items that may be used bysystem 10 in FIG. 1. For instance, in the present example where therotor blade of FIG. 3 is an item for which supplemental processes may beperformed, an identification number would be listed in column 78 for arotor blade type. Hereinafter, unless indicated otherwise, it will beassumed that item number 00-0001 corresponds to rotor blade 200 asillustrated in FIG. 3. In the exemplary database 17, item numbers00-0002 and 00-0003 correspond to a seat frame and a motor chassis,respectively.

Referring still to FIG. 7, image features column 86 lists at least oneand, in many cases, a plurality of feature subsets (each subsetincluding at least one feature) for each one of the identificationnumbers in column 78. For instance, column 86 lists six feature subsets82, 90, 91, 93, 95 and 97 for product ID number 00-0001 in column 78.The feature subsets 82, 90, 91, 93, 95 and 97 corresponding to image IDnumber 00-0001 include features F1 and F2, features F3 and F4, featuresF5 and F6, features F7 and F8, features F9 and F10 and features F11 andF12 which correspond to different edges of rotor blade 200 asillustrated in FIG. 3. Here, in addition to identifying features ofinterest that should be visually discernable within images to beobtained, it is contemplated that instructions for either processor 21or server 13 will also be specified in column 86 or elsewhere thatindicate how the features should be identified in obtained images. Forinstance, referring again to FIGS. 3, 5 and 7, instructions foridentifying top and bottom edges F7 and F8 may simply cause processor 21or server 13 to identify the most distal edges within an obtained imageor to identify the length dimension generally as the long dimension ofthe product and then identify the edges that define the length.

Here it is contemplated that the instructions for identifying featureswould be provided or specified during some type of commissioningprocedure. For instance, in at least some cases an HHD 20 may be used toobtain an image of a product where the image includes the features(e.g., edges, curvatures, etc.) of interest. Next, the obtained imagemay be presented via display 22 and interface tools (e.g., a mouse,trackball, etc.) may be used to select image features such as edges.Here, software run by server 13 may help the user charged withcommissioning to distinguish features. For instance, where a mousecontrolled cursor is moved to edge F7 (see again FIG. 3), the softwaremay identify the edge from differences in color in the image andhighlight the border or edge F7 for selection via the mouse inputdevice. Once a feature is selected in a commissioning image, thesoftware may automatically identify a specific region of interest withinthe obtained image in which the feature resides and develop and storerules for identifying similar features in subsequent images.

Referring again to FIG. 7, feature function column 92 indicates adifferent function for each one of the feature subsets listed in column86. For instance, a first function 94 indicates that a thickness T1defined by the space between features F1 and F2 is to be measured.Similarly, characteristic 96 indicates that a middle thickness T2between features F3 and F4 should be measured. The characteristicsassociated with feature subsets 91, 93, 95 and 97 indicate that a bottomthickness T3, a length L, a width W and a curvature C should bemeasured, respectively. In addition to indicating that specificdimensions should be identified, in at least some cases the featurefunction column will also indicate a desired value for a dimension or adesired value range and may instruct either the HHD processor 21 orserver 13 to perform some additional function such as, for instance,storing measured values, comparing measured values to required valuesand indicating variances, comparing measured and required/expectedvalues and indicating when measured values are within required ranges,etc.

Here again it is assumed that the feature functions are specified duringa commissioning procedure. For instance, after features of interest havebeen identified and rules for identifying the features in obtainedimages have been developed and stored, server 13 may provide aninterface for grouping image features into subsets (e.g., 82, 90, 91,etc) and for selecting associated feature functions. For example,referring again to FIGS. 3 and 7, after edge sections F1 and F2 areselected (e.g., subset 82), the commissioning software may providefunction options including “Measure dimension and provide feedback viadisplay”, “Measure dimension and store”, “Measure dimension, compare torange of acceptable dimensions and indicate differences”, etc. When afeature function is selected in this example, it is contemplated thatserver 13 would populate an associated section of feature functioncolumn 92. Other commissioning procedures are contemplated.

Hereinafter, unless indicated otherwise, it will be assumed that HHDprocessor 21 performs most of the supplemental processes. Nevertheless,it should be appreciated that, in at least some cases, some or most ofthe process steps could also be performed by server 13 in communicationwith HHD 20 via access point 16.

Referring now to FIG. 8, an exemplary method 136 according to at leastsome aspects of the present invention is illustrated. Referring also toFIGS. 2, 3, and 7, at block 138, a database 17 is provided thatcorrelates item IDs, visually distinguishable image features of interestto be identified in obtained images and feature function information. Atblock 140, an HHD 20 is provided that includes a camera/tag reader asdescribed above. At block 142, ID tags are provided on items to be usedwith HHD 20. At block 144, HHD 20 is used to obtain ID information froman ID tag (e.g., 202, in FIG. 7). At block 146 HHD processor 21 accessesdatabase 17 to identify features of interest and feature functionscorrelated with the obtained product identification information. Atblock 148, HHD 20 is used to obtain an image of the item from which theidentification information was most recently obtained. At block 150,processor 21 identifies the item features of interest in the obtainedimage. At block 152, processor 21 ascertains feature characteristics forcomparison to expected characteristics or to a range of expectedcharacteristics. Consistent with the example above, processor 21 mayattempt to identify any of thicknesses T1, T2 or T3, length L, width Wor curvature C.

Continuing, at block 154, processor 21 compares image characteristics torequired characteristics to identify any differences. At block 156,processor 21 provides feedback regarding feature characteristics. Here,in at least some embodiments, it is contemplated that to providefeedback, processor 21 transmits information via transceiver 34 toaccess point 16 and on to server 13 which provides feedback via displayscreen 22. In at least some cases, after block 156, either processor 21or server 13 may be programmed to correlate and store itemidentification information, features and characteristic informationincluding any differences between measured and required characteristicsin the results database 19 (see again FIG. 1).

While the systems described above have several advantages, such systemsalso may have some short comings in the context of certain applications.For example, in general, the systems described above rely on theassumption that an HHD user will be extremely familiar with the productsbeing imaged, the supplemental functions or processes being performedvia the system and advantageous relative juxtapositions (e.g., angle,spacing, pitch of HHD, etc.) between the HHD and the products/featuresbeing imaged from which to obtain the images. Indeed, to obtain an imageincluding features of interest for performing the supplementalfunctions, the HHD user would have to know which features are ofinterest and generally optimal angles and distances of the HHD to theproduct being imaged to ensure that suitable images are obtained foraccurately identifying the features and feature characteristics. Here,while filtering and software compensation schemes may be used tocompensate for minor differences between angles and distances of thecamera to the features of interest, larger differences may not beaccurately discernable and hence may not be able to be accuratelycompensated. Where an HHD user is not familiar with products,advantageous imaging juxtapositions and product features of interesthowever, the systems described above would be difficult to use at best.

To deal with the alignment problem described above, in at least somecases it is contemplated that system 10 may be programmed to help an HHDuser position HHD 20 such that an image obtained thereby is optimal forperforming other functions such as dimension measurement, coloridentification, and so on. To this end, referring to FIG. 9, in at leasesome cases an expanded product/function database 17′ may be providedthat includes, in addition to ID number column 78, features column 86and feature function column 92, an optimal images column 76. Asillustrated, optimal images column 76 lists at least one and, in somecases, several optimal images, for each one of the item ID numberslisted in column 78. For instance, an image identifier 73 which islabeled “I-1 (30° front view)” is listed in column 76 for itemidentification number 00-0001 in column 78. As another instance, twoseparate image identifiers are listed in column 76 for itemidentification number 00-0002, twelve separate image identifiers arelisted in column 76 for ID number 00-0003, and so on.

Here, it is assumed that an optimal image of a product has been capturedduring a commissioning procedure and stored in database 17 for each ofthe image identifiers listed in column 76. The idea here is that, whenHHD 20 is used to obtain an image of a product, the obtained image canbe compared to one or more optimal images to identify HHD movementsrequired to move the HHD into an optimal juxtaposition with respect tothe product being imaged or at least to identify if the HHD is within anacceptable range of optimal relative juxtapositions. In this case, anHHD position within an optimal range of relative juxtapositions willinclude a position wherein the characteristics that can be calculatedfrom an image obtained from the position will be sufficiently accuratefor the application associated with the characteristics. Here,sufficiency of accuracy is a matter of designer choice.

Feedback can be provided either via the HHD or some external device(e.g., display 22, see again FIG. 1) to help the HHD user reposition theHHD for obtaining a more optimal image for use in performing othersupplemental functions. Here, it is assumed that a guidance program canbe located in HHD memory device 25 that can compare obtained images tooptimal images, identify differences and generate relatively simpleguidance instructions usable by an HHD user to realign the HHD with anitem being imaged. In addition, the guidance program can recognize whenan obtained image is optimal or at least relatively optimal and can thenuse that image to perform other functions. For instance, exemplarysimple guidance instructions may indicate that an HHD is aligned ormisaligned with an item being imaged and may also indicate when anoptimal image has been obtained. As another example, the HHD may becapable of indicating that the HHD should be moved up, down, left,right, forward and in reverse in addition to indicating whether or notthe top of the HHD should be moved forward or backward or left or rightto affect pitch and roll adjustments of the HHD. Other more complex andhelpful active guidance systems are contemplated as describedhereinafter.

Referring again to FIG. 9, as indicated above, in at least some casesmore than one optimal image may be associated with a single product. Forinstance, an edge view image associated with identifier 101 and a planview associated with identifier 103 are specified for a rotor bladehaving ID number 00-0002. Where two or more optimal images areassociated with one product, it is contemplated that HHD 20 may beprogrammed to identify the optimal image most like an obtained image andprovide guidance toward obtaining an image akin to the most similaroptimal image. Here it is contemplated that the supplemental functionsassociated with each different optimal image would be different. Forinstance, as seen in FIG. 9, functions including calculating T1, T2 andT3 are associated with optimal image identifier 101 while functionsincluding calculating L, W and C are associated with optimal imageidentifier 103.

Referring now to FIG. 10, an exemplary simplified feedback configuration32 a is illustrated including three light emitting diodes (LEDs) 36, 38and 40 that are arranged on the top surface of housing barrel 29 withineasy view of an HHD operator. The LEDs are labeled as “misaligned”,“aligned” and “image obtained” indicating misalignment, alignment andthat an optimal image has been obtained. When HHD 20 is being used toobtain an item image, if the obtained image is not substantially similarto an optimal image, misaligned LED 36 is illuminated. Once an obtainedimage is substantially aligned with an optimal image, aligned LED 38 isilluminated and misaligned LED 36 is turned off. After an optimal imageor at least an image that is suitable for performing feature functionsassociated therewith has been obtained, LED 40 is illuminated.

Referring now to FIG. 11, an exemplary method 160 according to at leastsome aspects of the present invention where active guidance is providedto an HHD 20 user is illustrated. Referring also to FIGS. 1, 2, 3, 9 and10, at block 162, a database 17′ that correlates item IDs, optimalimages, visually distinguishable image features and feature functions isprovided. At block 164, an HHD including a camera/tag reader isprovided. At block 166, ID tags are provided on items to be used withsystem 10. At block 168, HHD 20 is used to obtain ID information fromone of the item ID tags. At block 170, HHD processor 21 accessesdatabase 17′ to identify optimal images associated with the obtained IDinformation. Consistent with the example above, where information hasbeen obtained from tag 202 on blade 200, processor 21 identifies theoptimal image associated with image identifier 73.

Continuing, at block 172, HHD 20 is used to obtain an intermediate imageof blade 200. At block 174, processor 21 compares the compared andoptimal images to identify the optimal image that is most similar to theobtained image. In the present case, because there is only one optimalimage in column 76 associated with the rotor blade identification number00-0001, the single optimal image associated with identifier 73 isidentified at block 174. In other cases where two or more optimal imageidentifiers are listed in column 76 (e.g., in the case of blades have IDnumber 00-0002), processor 21 selects one of the optimal images forperforming the guidance process at block 174.

After block 174, at block 176, processor 21 determines whether or notthe obtained image is substantially similar to the most similar optimalimage (e.g., the obtained image is suitable for sufficiently accuratelyperforming the feature functions associated therewith). Where theobtained image is not substantially similar to the optimal image,control passes to block 178 where processor 21 provides guidance via thefeedback devices (e.g., the LEDs 36, 38 and 40 in FIG. 10). After block178, control passes back up to block 172 where the subprocess includingblocks 172, 174 and 176 is repeated.

Referring once again to block 176, once the obtained image issubstantially similar to one of the optimal images, control passes fromblock 176 to block 179 where the most recent intermediate image is usedas a final image. In addition, feedback indicating alignment may beprovided via LEDs 36, 38 and 40 (see again FIG. 10). At block 180,processor 21 again accesses database 17′ to identify features ofinterest that should be visually distinguishable in the obtained image,the functions to be performed on the features and the requiredcharacteristics that are correlated with the item identification numberand matching optimal image. Again, in the present example, processor 21identifies features F1 through F12 as of interest and dimensions T1, T2,T3, L, W and C to be measured. At block 182, processor 21 attempts toidentify the features of interest in the obtained image. At block 183,processor 21 uses the identified features to calculate thecharacteristics that are to be compared to the expected characteristics.At block 184, processor 21 compares the calculated characteristics tothe expected characteristics to identify differences. At block 186,feedback regarding any differences is provided.

Here, once again, in at least some embodiments, feedback will beprovided by transmitting feedback information to access point 16 and onto server 13 where server provides the feedback information via display22. In other embodiments feedback may be provided via an audio speaker39 (see again FIG. 1) linked to processor 21. Here, where all calculatedvalues are within acceptable ranges, processor may generate a sounddiscernable as an affirmation of the product dimensions whereas, if oneor more values are outside acceptable ranges some other discerniblesound may be generated. In the alternative, one or two additional LEDscould be provided as part of feedback assembly 32 a (see again FIG. 10)to indicate acceptable or unacceptable calculated values.

Referring now to FIG. 12, a second exemplary feedback arrangement 32 bis illustrated which includes ten separate arrow shaped covers providedover LEDs to provide active guidance to an HHD user for obtaining anoptimal image. Here, the arrows are divided into two groups, one toindicate desirable HHD position movements within a space 44 and a secondgroup to indicate desirable HHD orientation movement provided within aspace 46. The arrows within position space 44 include an up arrow 52, adown arrow 56, a left arrow 54, a right arrow 50, a forward arrow 60 anda back arrow 62. As the labels imply, the position arrows 52, 56, 54,50, 60 and 62 are used to indicate that an HHD user should move an HHDup, down, left, right, forward and backward with respect to an itembeing imaged, respectively. For instance, when an item being imaged isto the right of the center of the HHD camera field of view, arrow 50indicating that the HHD should be moved to the right may be illuminated.Similarly, if an item appears much smaller in an obtained image than isoptimal, arrow 60 may be illuminated to indicate that the HHD should bemoved forward with respect to the item.

Similarly, the orientation arrows indicate how the barrel of the HHD 20should be moved to adjust pitch and roll. For instance, when arrow 66 isilluminated, the HHD barrel 29 should be moved forward with respect tothe HHD handle 27 thereby affecting a forward pitch movement. Similarly,where arrow 68 is illuminated, the barrel 29 should be rotated to theright with respect to handle 27 affecting a roll of the HHD.

In at least some other embodiments it is contemplated that a feedbackarrangement may include a small flat panel display screen mounteddirectly on HHD 20. To this end, an exemplary HHD display screen 32 c isbest illustrated in FIG. 15 and may include text instructions 228 toguide an HHD user to obtain an optimal image as well as an actual realtime video display section 224 showing a most recently obtained image226 and, in at least some cases, a phantom image 222 overlaid over theactual image to show an optimal image.

Where an HHD 20 is equipped with its own feedback display arrangement 32c, relatively detailed instructions can be provided to an HHD user forobtaining optimal images. To this end, referring once again to FIGS. 1,2, 3 and 9 and also to FIG. 13, initially when HHD 20 is turned on,instructions 212 may be provided via display 32 c instructing HHD userto position the HHD lens 28 in front of an item identification tag(e.g., 202) to obtain identification information by pressing theactivator button 30. Next, after identification information has beenobtained, the HHD processor 21 identifies the item associated with theidentification information and provides feedback 216 as shown in FIG. 14that identifies the type of item as well as provides another instruction218 to direct the lens at the blade along a 30° front view trajectory asrequired to obtain the optimal image (see again image indicator 73 incolumn 76 of FIG. 9). Once an image of the item is obtained, asillustrated in FIG. 15, the image 226 is presented within image space224. In addition, an optimal image phantom 222 is provided within space224 to help the HHD user to quickly and intuitively determine how theHHD position has to be modified in order to obtain the optimal image. Inaddition, instructions 228 are provided by the processor 21 to indicatetextually how the HHD position has to be modified to obtain the optimalimage.

In the illustrated example, the instructions are provided in a step-wisefashion to first direct the HHD user with respect to pitch and roll,left and right and up and down movement and then to instruct the userwith respect to forward and backward movement. Thus, in FIG. 15 the textinstructions request that the user move the HHD to the right and rotatethe HHD clockwise for optimal alignment. In FIG. 16, once the operatorhas moved the HHD right and rotated the HHD counter-clockwise, a new setof instructions 232 request that the user move the HHD forward foroptimal alignment.

Referring to FIG. 17, after the HHD has been moved forward anappropriate amount and the obtained image 226 is substantially similarto the optimal image 222, processor 21 obtains a final image and usesthat final image to identify features of interest and to calculatecharacteristics of interest from the features of interest. Next, in atleast some applications, processor 21 compares the measuredcharacteristics to the required characteristics and provides feedback.As illustrated in FIG. 17, in the present example, processor 21indicates the length, width and three thicknesses of interest for theblade and indicates that, in the present case, the blade curvature isoutside the acceptable range.

Referring once again to FIG. 9, as described above, in at least somecases, two or more image identifiers in column 76 may be associated witha single item identification number in column 78. Again, in FIG. 9, edgeview image identifier 101 and plan view image identifier 103 areassociated with item identification number 00-0002 in column 78.

In at least some cases it is contemplated that where multiple optimalimages are associated with a single item identification number and anobtained image is similar to two or more of the optimal images,processor 21 may provide alternate instructions and allow the HHD userto obtain any one of the optimal images through different movements ofthe HHD 20. For the purposes of the next example, referring again toFIG. 3, it will be assumed that tag 202 corresponds to itemidentification number 00-0002 in database 17′ illustrated in FIG. 9 sothat both edge view identifier 101 and plan view identifier 103 areassociated with blade 200. Referring once again to FIGS. 1, 2, 5 and 9as well as FIG. 18, after identification information has been obtainedfrom a tag 202 on blade 200 as well as an image 226 of blade 200,processor 21 may identify either of the edge view or the plan viewimages associated with image identifiers 101 and 103 as possible optimalimages the HHD user is attempting to obtain.

In this case, processor 21 may be programmed to provide alternateinstructions 248 guiding the user to either align the HHD to obtain theoptimal edge view image or to obtain the optimal plan view image.Referring to FIG. 19, where the HHD user follows the instructions toobtain the optimal edge view image, once the optimal image has beenobtained, characteristic dimensions associated therewith includingthickness T1, T2 and T3 are obtained and feedback is provided regardingthe measured values 260 as well as feedback regarding whether or not thevalues are acceptable 262. Referring to FIG. 20, where the HHD userfollows the instructions for obtaining the optimal plan view image, oncethe optimal plan view image is obtained, feedback 272 is providedindicating the measured values as well as providing warnings whereappropriate.

In addition to the supplemental functions'described above, oneadditional and particularly advantageous supplemental function that maybe performed by system 10 includes a part verification function. To thisend, in many cases identification tags are provided on products or itemsso that product consumers can verify that the items are genuine partsfrom specific manufacturers that are known to provide high qualityproducts. Thus, for instance, to ensure that a rotor blade wasmanufactured by Harbinger Aero Parts, a reputable parts manufacturer,Harbinger Aero Parts may provide identification tags on each one oftheir rotor blades that can be used by an end user to attempt to verifyauthenticity. Unfortunately, part counterfeiters have begun to copy theinformation on identification tags and place similar tags on counterfeitparts so that end users can no longer verify part authenticity by taginformation alone.

To deal with the above problem, it has been recognized that a sourcemark in the form of a trademark or the like can be provided either aspart of an identification tag or proximate an identification tag suchthat when an image of the identification tag is obtained, an image ofthe source mark is also obtained. In this case, in addition to verifyingan item identification number in an attempt to verify authenticity, thesource mark can be compared to a trademark of the trusted supplier and,if the source mark is different than the trusted supplier's trademark,the part can be recognized as a counterfeit part. In addition, if thesource mark is not present in the obtained image, the part can berecognized as a counterfeit part. While part counterfeiters could copysource marks as well as identification tags and use those marks and tagstogether in an attempt to continue to counterfeit products, copying asource mark like a trademark would be a separate trademark violation andwould be relatively easily actionable.

Consistent with the comments above, referring to FIG. 21, an exemplarysource mark 300 is shown printed or placed on a blade 200 next to anidentification tag 202 where both the mark 300 and tag 202 can besimultaneously imaged for part verification purposes. Referring also toFIG. 22, an exemplary method 302 for verifying parts using a source markand an identification tag is illustrated. Beginning at block 304, adatabase is provided that correlates source marks with itemidentification information. At block 306 ID tags and source marks areplaced on items to be used with the system 10. At block 308, an HHD 20is provided that is capable of simultaneously reading ID tag and sourcemark combinations. At block 310, the HHD is used to read a tag andsource mark combination. At block 312, the database is used to verifysource mark authenticity. At block 314, the database is used to identifythe type of item associated with the identification tag. At block 316,the HHD processor 21 determines whether or not the source mark and itemtype have been verified. Where the source type and item type have beenverified, at block 318 processor 21 indicates that the part isauthentic. In the alternative, where either the source mark or the itemtype has not been verified at block 316, at block 320 the HHD processor21 indicates that the part is likely a counterfeit part. Indication maybe either audible (e.g., a small speaker) or visual (e.g., an LED or thelike).

In at least some cases it is contemplated that, after an HHD is used toobtain product identifying information from a tag that is placed on orassociated with a particular product, the HHD will continue to beassociated with the particular product for the purpose of performingsupplemental functions until a transition indication or event occurssuch as either new tag information being obtained from a different tagor the HHD user performing some process to indicate that the associationbetween the HHD and the product should be broken. Thus, for instance,after identification information is obtained from a tag, HHD 20 may beused to obtain ten different optimal images of the product associatedwith the identification tag, the HHD processor 21 performing a differentsubset of supplemental functions for each one of the obtained imageswithout having to reassociate with the product each time a new image isobtained. At any time, if the HHD is used to obtain information from adifferent ID tag, HHD 20 association with the previous product is brokenand a new HHD product association is formed with the new product.

Consistent with the comments in the previous paragraph, referring now toFIG. 23, another method 350 that is consistent with at least someaspects of the present invention is illustrated. Referring also to FIGS.1, 2 and 5, at block 352, a database is provided that correlates itemIDs, optimal images, guidance functions, visually distinguishable itemfeatures of interest and feature functions. At process block 354, a HHD20 including a camera/tag reader is provided. At block 356, ID tags areprovided on items or products. At block 358, HHD 20 is used to obtain IDinformation from an ID tag located on one of the products. At block 360,HHD 20 is used to obtain an image of the product including the tag fromwhich identification information was most recently obtained. At block362, HHD processor 21 performs supplemental processes associated withthe obtained identification information. At block 364, HHD processor 21determines whether or not new tag identification information has beenobtained. Where new tag identification information has not beenobtained, control passes down to block 366. At block 366, processor 21determines whether or not some other disassociating indication has beenreceived by the HHD. Here, for instance, a disassociating indication mayinclude holding the activation button 30 in for more than 5 continuousseconds. Where no other disassociating indication has been received,control passes back up to block 362 where the sub-loop including blocks362, 364 and 366 continues. Referring again to block 366, where adifferent disassociating indication is received, control passes back upto block 358 where the HHD processor 21 waits to receive new ID taginformation. Referring once again to block 364, where new tag IDinformation is obtained, control passes back up to block 360 whereprocessor 21 waits for a new image to be obtained via HHD 20.

While the invention may be susceptible to various modifications andalternative forms, specific embodiments have been shown by way ofexample in the drawings and have been described in detail herein.However, it should be understood that the invention is not intended tobe limited to the particular forms disclosed. For example, othersupplemental processes are contemplated. For instance, kit componentverification is contemplated. To this end, referring to FIG. 24, a kit420 is illustrated that includes a number of components where at leastone of the components includes a tag 422. Here, the tag information maybe associated with a list of kit components in a database where imagefeatures include instances of the kit components along with instructionsregarding how to identify the components in an image, the featurefunction being to identify each list component in an obtained image.When an image of a kit is obtained, the HHD processor 21 may beprogrammed to identify the tag in the image, obtain kit identificationinformation from the tag, identify the kit image and component list,attempt to identify the kit components in the image and then providefeedback regarding whether or not the kit is complete and any componentsthat do not appear in the kit.

In addition, systems are contemplated wherein a portable HHD may beassociated with an item in ways other than obtaining tag informationfrom a tag that resides on the item. For instance, in at least somecases a tag may reside in a parts booklet or the like where the tag isspatially associated (e.g., on the same page in the booklet) with animage of the item/product. For instance, a tag for a rotor blade asdescribed above may be presented next to an optimal image. Here, an HHDuser could obtain tag information from the booklet to associate the HHDtemporarily with an item and then perform image obtaining processes andother supplemental processes using the HHD as described above.Similarly, an end process or disassociation tag may be included in abooklet or elsewhere in proximity to where an HHD is used todisassociate the HHD with an item that the HHD is currently associatedwith.

Moreover, while some systems are described above as including guidancecapabilities, in at least some cases it is contemplated that no guidancefunctions may be provided. Similarly, while some systems are describedabove that include association via a tag, in some cases such associationmay not be supported. For instance, where an HHD is used with a singleitem type, the HHD may be preprogrammed for use with the single itemtype and the supplemental processes may all be the same regardless ofthe instance of the item that is imaged. Here there is still value inthe inventive concepts as different processes may be performed dependingon which image is obtained and depending on the quality of the imagesobtained.

Furthermore, various types of commissioning procedures are contemplatedwherein items having known standard characteristics are imaged togenerate at least one optimal image for each item and then features onthe items are identified as well as feature characteristics of interestand acceptable ranges of characteristic values. The present inventionmay be used with any type of commissioning procedure that generatessuitable database information.

In addition, while the inventive aspects have been described above inthe context of an HHD including a camera/sensor capable of obtainingboth ID tag information as well as images of products/items, it shouldbe appreciated that other HHD configurations are contemplated where thecamera and the tag reader are separate HHD components. Here, note thatthe tag reader may take several forms including a bar code reader, anoptical character recognition reader, an RF sensor, etc.

Moreover, instead of storing optimal images to facilitate guidance,other types of information that reflect optimal images may be stored.For instance, general orientation of edges of a product may be storedalong with ranges of dimensions for comparison to similar features anddimensions in obtained images.

Thus, the invention is to cover all modifications, equivalents, andalternatives falling within the spirit and scope of the invention asdefined by the following appended claims. To apprise the public of thescope of this invention, the following claims are made:

1. A method for use with a portable device including a processor, themethod comprising the steps of: providing an identification tag on atleast one item that includes item identifying information; obtainingitem identifying information from the identification tag via theportable device; obtaining at least one image of the at least one itemvia the portable device; and performing a supplemental process using theidentifying information and the at least one image.
 2. The method ofclaim 1 wherein the supplemental process includes correlating the imagewith the identifying information.
 3. The method of claim 2 wherein thesupplemental process further includes storing the correlated image andidentifying information in a database for subsequent use.
 4. The methodof claim 3 also for use with a processor linked to a database and areceiver wherein the portable device includes a transmitter, the step ofstoring including transmitting the correlated information to theprocessor via the transmitter and receiver and storing the correlatedinformation in the database via the processor.
 5. The method of claim 1wherein the at least one item includes at least one feature that isdistinguishable in the obtained image and the supplemental processincludes identifying the at least one feature in the image.
 6. Themethod of claim 5 wherein the supplemental process further includesidentifying at least a first characteristic of the identified feature.7. The method of claim 6 also for use with a database wherein at leastone expected first characteristic is stored, the supplemental processfurther including comparing the identified first characteristic with theat least one expected first characteristic and, when the identifiedfirst characteristic is different than the at least one expected firstcharacteristic, performing a warning function.
 8. The method of claim 7wherein the portable device includes at least one of an audible warningdevice and a visual warning device and wherein the step of performing awarning process includes activating the at least one of the audible andvisual warning device in a first way.
 9. The method of claim 8 whereinthe supplemental process further includes, when the identified firstcharacteristic matches the acceptable first characteristic, activatingthe warning device in a second way that is distinguishable from thefirst way.
 10. The method of claim 7 wherein the step of performing awarning function includes providing feedback that indicates thedifference between the identified first characteristic and the at leastone expected first characteristic.
 11. The method of claim 6 wherein thestep of identifying at least a first characteristic includes identifyingone of a dimension, a color, a pattern, a curvature, an angle withrespect to at least a second feature within the image and a relativejuxtaposition with respect to at least a second feature within theimage.
 12. The method of claim 11 wherein the supplemental processfurther includes correlating the item identifying information with theidentified characteristic and storing the correlated information in adatabase.
 13. The method of claim 5 further including the step of usingthe information obtained from the tag to determine the feature to beidentified in the image.
 14. The method of claim 13 also for use with aprocessor that has access to a database that correlates items withfeatures to be identified in associated images and wherein the step ofusing the information obtained from the tag to determine the featureincludes identifying a feature correlated with the identified item inthe database.
 15. The method of claim 6 wherein the step of identifyingat least a first characteristic includes identifying at least onedimension defined by the at least one feature and at least a secondfeature in the image.
 16. The method of claim 15 wherein the portabledevice further includes an output device and wherein the supplementalprocess further includes presenting the identified dimension via theoutput device.
 17. The method of claim 16 wherein the output device is avisual output device and wherein the step of presenting the identifieddimension includes presenting the dimension via the visual outputdevice.
 18. The method of claim 16 also for use with a database whereinat least one expected dimension is stored, the supplemental processfurther including comparing the identified dimension with the at leastone expected dimension and, when the identified dimension is differentthan the at least one expected dimension, indicating the difference viathe output device.
 19. The method of claim 18 wherein the step ofindicating the difference includes indicating the at least one expecteddimension.
 20. The method of claim 5 wherein the step of identifying theat least one feature includes identifying circumstances under which theimage was obtained and compensating at least in part for the identifiedcircumstances.
 21. The method of claim 1 further including placing asource mark on the item, the method further including the step ofobtaining an image of the source mark.
 22. The method of claim 1 whereinthe step of obtaining an image includes obtaining an image including theat least one item and several other items proximate the at least oneitem.
 23. The method of claim 1 wherein the portable device includes analignment device useable to align the portable device with the at leastone item, the method further including the step of, prior to obtainingthe at least one image, using the alignment device to align the portabledevice with the at least one item.
 24. The method of claim 1 also foruse with a display device and a receiver separate from the portabledevice wherein the portable device includes a transmitter, thesupplemental process further including transmitting the obtained imageto the display device via the transmitter and the receiver anddisplaying the obtained image via the display device.
 25. The method ofclaim 5 wherein the step of identifying the at least one featureincludes identifying at least one region of interest (ROI) within theimage that should include the at least one feature and processing theportion of the image in the ROI to identify the at least one feature.26. The method of claim 1 wherein the supplemental process includes atleast a subset of storing the image, storing the obtained identifyinginformation, storing the image correlated with the obtained identifyinginformation, identifying characteristics of obtained images, correlatingand storing identified image characteristics with obtained identifyinginformation, comparing identified image characteristics with expectedimage characteristics to identify differences, indicating differencesbetween identified image characteristics and expected characteristics,storing differences between identified image characteristics andexpected characteristics, providing guidance for aligning the portabledevice to obtain an optimal image and using the obtained image toidentify at least one optimal image that is most similar to the obtainedimage.
 27. The method of claim 1 wherein the step of obtaininginformation from the tag includes one of reading the information fromthe tag and exciting the tag to transmit information and receiving thetransmitted information.
 28. The method of claim 1 wherein the tag isone of a bar code, an RFID tag, an optical character tag and a dotmatrix tag.
 29. The method of claim 1 wherein the supplemental functionincludes using the image content to discern at least one feature to beidentified in the image.
 30. The method of claim 1 wherein the portabledevice has access to a database that stores at least first and secondoptimal images of the at least one item that are correlated with itemidentifying information and, wherein, the supplemental process includescomparing the obtained image to at least one of the first and secondoptimal images to identify the optimal image most similar to theobtained image.
 31. The method of claim 30 wherein the database furtherincludes at least first and second distinct sub-processes associatedwith the first and second optimal images, respectively, and, wherein,when the obtained image is most similar to one of the optimal images,the supplemental process includes performing the sub-process associatedwith the most likely associated optimal image.
 32. The method of claim31 wherein each of the at least first and second optimal images includeat least one distinguishable feature and wherein the sub-processesinclude attempting to identify the distinguishable feature in theobtained image.
 33. The method of claim 32 wherein the distinguishablefeature in the first optimal image is different than the distinguishablefeature in the second optimal image.
 34. The method of claim 1 whereinthe at least one item includes at least two features that aredistinguishable in the obtained image and the supplemental processincludes identifying the at least two feature in the image.
 35. Themethod of claim 34 wherein the supplemental process further includesidentifying a plurality of characteristics of the at least two featuresin the obtained image.
 36. The method of claim 35 wherein thesupplemental process further includes providing feedback regarding atleast a subset of the plurality of features.
 37. The method of claim 36wherein the step of providing feedback includes providing feedback viathe portable device. 38-57. (canceled)